Repository navigation
data(catalog): rebuild the catalog database with the catalog-audit fixes (#670) — squash-merge only - #697
Merged
Conversation
Corwin's NGC notes: "NGC 771 = 50 Cassiopeiae is a star" (an A1 V star, no nebulosity on the POSS1). 50 Cas is HIP 9598, IAU name Gang, which #695 lists as Nam 9598. The listing joins that star; the match guard accepts the join (both type *, 0.04' apart). Refs #670 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The one rebuild for the catalog-audit campaign. It carries the import, loader, star-identity and correction changes of #671 to #696, and the NGC 771 correction before it. Built from a fresh file with `python -m PiFinder.catalog_imports.main`; two builds with different PYTHONHASHSEED values are byte-identical: sha256 03b6bcb3288b1a2909e4fedb49559f46517c7bf4e3f13ad097f848f2c5ad670e All 225 corrections apply (70 of them hip identifications). Refs #670 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…in fixes test_catalog_data.py reads the committed database, so its assertions land with the rebuild: 22 catalogs, EGC 102, SaM 2157 (rejected pairs skipped), Nam 578, and IC 1805 typed C+N (Trumpler "n", #676). New assertions, each read from the rebuilt file: the M 82 and M 57 preferred names; a not-found NGC object's description; Mizar as one D* sky object with its Str, SaM, WDS and Nam listings and the double-star filter magnitude; Vega's V 0.03 and Hipparcos position; SaM 1019 on HIP 44804; IC 2145 joined to NGC 2086, NGC 771 to Nam 9598, and Gumala one object (WDS 99456 + Nam 94645); Dhanuśarāgra with its diacritics. docs/ax/catalog.md §9: the status note goes, since the committed database now matches the import §9 describes. Its code map and the ADR 0046 link stay, as a plain paragraph. The compressed size is about 24 MB. Refs #670 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… text
load_collinder added the raw text of collinder.txt's "NGC/Other Cat." column
and of collinder2.txt's "Other names" column as names. "581 (M103)" stayed a
plain name, so Col 14 was a second sky object 0.1' from M 103, and Col 399
carried "4-5 Vul." and the one name "Al Sufi's Cluster, Brocchi's Cluster,
Coathanger".
The loader now reads both columns (collinder_names):
- a designation is written out: "581 (M103)" is "NGC 581" and "M 103",
"IC1848" is "IC 1848", "Tr.9" is "Trumpler 9". 29 Collinder listings now
join the Messier, NGC or IC object that Collinder names (26 Messier
clusters, Col 334 = NGC 6374, Col 382 = IC 4725, Col 26 = IC 1805);
- "(incorrectly) 3247" gives no name, so Col 220 can never join NGC 3247;
- "nl", the star numbers "4-5 Vul." and "9-12 Gem", and notes in
parentheses ("(PMC)", "(Mel 72 per A/H)") are no names;
- "2384a" is the name "NGC 2384a", which joins nothing;
- a comma list is separate names, and a name is listed once.
The IC 2602 correction removed "Theta Carinae Cluster, Southern Pleiades".
The loader now splits that list, so the correction is deleted.
Refs #670
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A fresh-file build of this branch (the database was deleted first), after the Collinder loader fix. Two builds with PYTHONHASHSEED=1 and 4242 are byte-identical: sha256 c3cdb493338f57a4c01d8d69b4170039b8dffedf991b4b93ce73ed961dad861d The import loads and applies all 224 corrections. Against the first rebuild of this branch (ead7120, sha256 03b6bcb3...): 146,631 -> 146,602 sky objects, because 29 Collinder listings now share the Messier, NGC or IC object that Collinder names. The 151,754 catalog listings are the same. No other join changed. This replaces ead7120's database. Squash-merge the pull request, so that only this version of the file lands on main. Refs #670 Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…base
Col 399 has "Al Sufi's Cluster", "Brocchi's Cluster" and "Coathanger" as
three names, and no "4-5 Vul.". Col 14, 189, 382, 26 and 334 share the
Messier, NGC or IC object that Collinder names; M 44 keeps "Beehive Cluster"
first, and IC 1805 keeps its type C+N. Col 220 is not NGC 3247, and Col 142
("NGC 2384a") is not NGC 2384. No Collinder name holds raw column text.
Refs #670
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brickbots
added a commit
that referenced
this pull request
Nov 16, 2026
…nd the release-branch plan The playbook was written on 2026-09-05. Since then #657 and #658 landed, and parts of it became wrong. One command was wrong from the start. The conflict check piped `git merge-tree` into `grep -c '<<<<<<<'`. The two-argument form prints a tree id and a file list, never conflict markers, so the count is always 0. It is 0 on the 2.6.3 cut, which had a version.txt conflict. The check now uses `--write-tree` and the exit code, and compares the result with main's tree. The NixOS section repeated mechanics that belong to nixos/RELEASE.md on the nixos branch, and four statements no longer matched the code: the gate flag (now nixos_migration), the cache host, the reason for the tarball size limit, and who creates the tag. The section is now a pointer. Two checks could not pass: ".github byte-identical" (the difference is part of the cut) and "ADR ledger has no gaps" (0036 to 0045 are on the NixOS line). The first now names the real hazard: release is the default branch, and its old release.yml writes the manifest key that 2.6.3 reads. New sections: - Between cuts: the rule for promoting docs from main to release, and the fact that a promotion also carries code (#639 and #640 reached release after the v2.6.3 tag with no version bump). - Where the tracks meet: 2.6.4 is the first Pi OS release that can migrate to NixOS. The contract that freezes at the cut, the 7x SQUARE path that ignores the gate, the 90-day life of a beta, and opening the gate as a release of its own. - Decided for 2.6.4: #697 ships in it, and no stable NixOS release comes first. - A transition plan for the release branch: it stays the Pi OS line for ever, the default branch moves to main after the 2.6.4 cut, the cut from main ends when #379 merges, and /en/release/ redirects to /en/stable/ at the first stable 3.x. Docs snapshots: /en/v2.6.3/ exists, so the text no longer says nothing exists for 2.6. The "previous versions" notes were not updated for it; that step now belongs to the prep PR of the next release. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
brickbots
added a commit
that referenced
this pull request
Nov 19, 2026
…SE.md (#644) release_notes/RELEASE-PLAYBOOK.md is the process behind a release, pulled out of the 2.6.1, 2.6.2 and 2.6.3 plans so the next cut starts from it. - Phases 1 to 8 for a 2.x release, with the traps that bit earlier cuts. - The rule for promoting documentation fixes from main to release between cuts. - A pointer to the NixOS release process (nixos/RELEASE.md on the nixos branch), not a copy of it. - The migration, where the two lines meet: 2.6.4 is the first Pi OS release that can migrate to NixOS. What freezes at that cut, and the checks before the migration gate opens. - Decided for 2.6.4: #697 ships in it, and no stable NixOS release comes first. - A proposed transition plan for the release branch: it stays the Pi OS line, the default branch moves to main after the 2.6.4 cut, the cut from main ends when #379 merges, and /en/release/ redirects to /en/stable/ at the first stable 3.x. RELEASE.md at the repo root was a byte-identical copy of release_notes/2.6.1.md with nothing referencing it. Removed. Phase 6 (the 2.x SD image) and the announcement channels in phase 7 are still stubs. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…lease notes link to Refs #670 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… it dubious The D* rule's WDS clause now types a WDS listing D* if any of its pairs has a companion magnitude and is not noted X (dubious). Optical pairs (S, U, Y, L), pairs without the primary (BC), and pairs of any separation or magnitude difference count. The type says what the object is; visibility is for the filters (a double filters by its brighter star) and for the separation and magnitudes the entry shows. This follows mrosseel's review of #697. The old limits (0.5-60", 5 mag, primary only, no optical flags) typed 40,788 of 131,303 WDS listings single; the new rule types 10,597 single. The Vega (SaM 1241) and Fomalhaut (SaM 1633) obj_type corrections are deleted: they undid the old rule for optical/wide companions, and both stars are now D* by WDS, as SAC lists them. zeta Scl's (SaM 1745, only pair X-flagged) agrees with the new rule and stays. 222 corrections. The type still affects no match: the match guard treats * and D* alike (is_star_type), and the star identity reads the type only to type the merged star. Refs #670 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every WDS pair is a double star unless WDS marks it dubious (X) or gives no companion magnitude. Objects typed D*: 91,444 -> 121,232; WDS listings D*: 90,916 -> 120,706. Only types change against the previous build (c3cdb493...): 30,049 listings go from * to D*, no join, value, name or image changes. 222 corrections apply clean. Two fresh builds (PYTHONHASHSEED=1 and 4242) are byte-identical: sha256 474d0331a5bfacebb90b2f2cfd08c1e1db1ca24ac18efc92459aedecf35a52c7 Refs #670 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tabase Atria (optical pair WFC 179) and Suhail (SEE 109, 12.6 mag fainter) are D*; Arcturus (only pair HDS 2003, flagged X) is *; Vega is D*. WDS listings by type: 120,706 D*, 10,597 *. The 2.6.4 catalog release note gets a "Double stars follow WDS" bullet. Refs #670 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 19, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Squash-merge only: the branch has three database commits, and only the final one must land.
This pull request replaces PiFinder's catalog database with a new build. The new database has the fixes from the catalog check in #670. Objects have better names, positions, types and magnitudes. A star that many catalogs list is now one object, with one position from Hipparcos. A new catalog, Named Stars, lists every star that has an official IAU name. This is the only database rebuild for PRs #671 to #696, because each committed version of the database is a download of about 24 MB for every PiFinder that updates.
The pull requests that this rebuild carries:
This pull request also adds one correction: NGC 771 is the star 50 Cassiopeiae (Corwin), so it joins Nam 9598 (Gang).
It also corrects how the import reads the Collinder catalog's names (see "Collinder names" below). The first database of this pull request had names such as "581 (M103)", "nl" and "4-5 Vul." on Collinder objects. The database was built again after that correction.
It also changes the rule that types a WDS star as a double star (see "WDS double-star rule" below). Now every pair that WDS lists makes a double star, unless WDS marks the pair as dubious. The database was built again after that change. It ships in release 2.6.4.
Fixes #670
Fixes #668
Fixes #667
Commits
fix(catalog-corrections): NGC 771same_as = "Nam 9598"insteinicke.toml. The source is Corwin's NGC notes: "NGC 771 = 50 Cassiopeiae is a star". The match guard accepts the join (both type*, 0.04′ apart), so it has noguard = false.data(catalog): the first rebuild ofastro_data/pifinder_objects.db(sha25603b6bcb3…). Commit 6 replaces it.test(catalog-data):tests/test_catalog_data.pyanddocs/ax/catalog.md§9.main(feat(migration): Pi OS side of the NixOS migration, with a btrfs root and kept settings #657, ci(release): the release workflow of the nixos branch, with migration_v2 keys #658, fix(display): run the 1.5" OLED at 20 MHz SPI #701). None of them changes the catalog: a build after the merge was still03b6bcb3….fix(catalog-import): the Collinder loader reads its two name columns (collinder_names), with tests. One correction that the loader now makes unnecessary is deleted.data(catalog): the second rebuild ofastro_data/pifinder_objects.db(sha256c3cdb493…). Commit 11 replaces it.test(catalog-data): assertions for the Collinder names and joins.main(fix(sqm): measure the crop, and scale by the delivered analogue gain #666, the SQM fix). It changes no catalog file. A build after the merge, with a third hash seed, is stillc3cdb493….docs(release-notes):release_notes/2.6.4-catalog.md, a one-page summary of the catalog changes for users. The 2.6.4 release notes link to it.fix(catalog-import): the WDS double-star rule (is_double_star_pairinwds_loader.py), with tests, anddocs/ax/catalog.md§9.3 and the CatalogCONTEXT.md. Twoobj_typecorrections that the new rule contradicts are deleted (Vega and Fomalhaut).data(catalog): the finalastro_data/pifinder_objects.db(sha256474d0331…), in its own commit.test(catalog-data): assertions for the WDS double-star rule, and a release-note line.The branch has three commits that change the database file (2, 6 and 11). A squash merge puts only the final file on
main. A merge commit would put all three versions in the history, and every PiFinder would download all three.Collinder names
The Collinder catalog comes from two files.
collinder.txthas a column "NGC/Other Cat.", andcollinder2.txthas a column "Other names". The loader added the text of each column as one name, without reading it. This is an error in the loader, so the loader is corrected (ADR 0046); no correction file is used.collinder.txt, column "NGC/Other Cat." (471 rows):Text in parentheses is kept only if it is one designation ("(M103)", "(Harvard 8)"). All other text in parentheses is a note ("(PMC)", "(Mel 72 per A/H)").
collinder2.txt, column "Other names" (471 rows):29 Collinder listings now share the Messier, NGC or IC object that Collinder names. Before, each of them was a second sky object at almost the same position, because the import could not read "581 (M103)" as a designation. The join uses the same rule as all other catalogs (#671, #680): the designation must have a known catalog prefix, and the match guard must accept the position and the type. The match guard accepted all 29.
A build before the loader change and a build after it were compared, listing by listing:
C+N(fix(catalog-import): type Steinicke rows by status, attach duplicates, mark not-found objects #676).collinder.toml. It removed the name "Theta Carinae Cluster, Southern Pleiades" from IC 2602. The loader now reads that text as two names, so the import has 224 corrections, not 225 (222 after the WDS rule change below).WDS double-star rule
mrosseel, who observes double stars, reviewed the catalog changes. His principle: if WDS lists a pair, it is a double star. Whether you can see the companion is a different question. The filters answer it (a double star filters by its brighter star, #692), and so do the separation and the two magnitudes that each entry shows. Rich accepted this principle.
The rule before. A WDS listing was a double star (
D*) only if one of its pairs agreed with all of these conditions:All other WDS listings were single stars (
*).The rule now. A WDS listing is a double star if one or more of its pairs agrees with these two conditions:
Thus optical pairs are double stars. Pairs without the primary star (BC, CD) are double stars. There is no limit for the separation or for the magnitude difference.
c3cdb493…)474d0331…)D**D*sky objectD**The old committed database (release 2.6.3) typed every WDS listing
D*, and it did not merge stars. Of the 10,597 WDS listings that are now*: in 9,726, WDS marks every pair as dubious. In 763, no pair has a companion magnitude. In 101, both are true. In 7, each pair is one or the other.Against the old rule, only types change. A comparison of the two builds, listing by listing, finds no join changes, no name changes and no image changes. 30,049 listings change from
*toD*and no listing changes fromD*to*: WDS 29,790, Nam 178, SaR 35, Str 24, TLK 18 and SaM 4. The type has no effect on a match: the match guard accepts*andD*as the same kind, and the star identity step uses the type only to type the merged star. The star identity step merges the same 2,580 sky objects into the same 1,969 stars as before. The WDS loader's DM numbers, its descriptions and its magnitudes do not use the rule.Examples that are now
D*:Vega and Fomalhaut had corrections that typed them
*, because SAC lists them but their companions are field stars. These corrections contradict the new rule, so they are deleted. The import now has 222 corrections.Arcturus stays
*: its only WDS pair, HDS 2003, is marked dubious. ζ Scl keeps its correction to*: its only WDS pair, B 631, is marked dubious, so the correction agrees with the new rule.What a user sees: the double-star type filter shows more pairs than in the two earlier databases of this pull request. Each entry shows the separation and both magnitudes, so you can decide if you can split the pair.
Data effect
D*gzip -9size (bytes)Listings and sky objects per catalog:
A lower count of sky objects in a catalog means that some of its listings now share a sky object with each other (NGC and IC duplicates, double-star rows of one system). The count of all sky objects is also lower by 29 than in the first database of this pull request (146,631), because of the 29 Collinder joins.
Stars merged. The star identity step merges 2,580 sky objects into 1,969 stars (70 of them by a cited
hipcorrection). Before, only 1 sky object had listings in two or more star catalogs. Now 1,961 do. A star that no double-star list (SAC, RASC or WDS) lists as a double is typed*, notD*(#683; see "WDS double-star rule").Names. There are 1,207 new name strings and 733 removed ones.
Removed listings
Only 5 listings of the old database are not in the new one. All 5 are SAC Doubles rows that SAC marks as a rejected pair ("rej."). Since #683 the import skips these rows. Their sequence numbers stay taken, so every other SaM number names the same row as before. Each star is still in the database through its WDS listing:
The new listings are EGC 92 to 102 and the 578 Nam listings. No other catalog code and sequence number was added or removed.
What a user sees for a removed listing: an observation of SaM 149 stays in the observation log and in its export, but no object shows as observed. An observing list entry "SaM 149" opens from its coordinates, if the list has them.
Listings that moved. 151,165 listings are in both databases. 1,508 of them are now more than 5′ from their old position:
An observation of such a listing now opens the corrected position.
Compatibility on a PiFinder that updates
The rebuild changes the sky object ids (
objects.id). Nothing outsidepifinder_objects.dbstores one:observations.db,obs_objects)observations_db.py, the table'scatalogandsequencecolumns)._resolve_object_idsfinds each listing's sky object in the current database at startup, and keeps the result in memory only.obslists/)obslist_formats.py). They are read back by designation, then name, then coordinates.state.py). It is not saved.M82_POSS.jpg), then from names (cat_images.py).cache/catalogs/composite_objects.pkl)config.jsonThe observation log stores the listing, and PiFinder shows "observed" for the sky object. So an observation of Col 14 now also shows M 103 and NGC 581 as observed, and an observation of M 103 shows Col 14 as observed. This is the same for the other 28 joined Collinder listings.
Reproducibility
The database was deleted first, then built with
python -m PiFinder.catalog_imports.main. Two builds withPYTHONHASHSEED=1andPYTHONHASHSEED=4242are byte-identical:The import loads and applies all 222 corrections (70 of them
hip). It reports no stale, redundant or conflicting correction. The build takes about 45 seconds.PRAGMA integrity_checkisok. The earlier databases of this pull request had sha25603b6bcb3…andc3cdb493…; they are not the final file.Findings
The catalog check's
findings.py(paperChart, read-only) against each database:c3cdb493…)Against the old committed database, 409 findings are gone and 166 are new. The first database of this pull request had 1,016 findings, the same as #694's final test build. The Collinder change removes one more: IC 2602 now has the name "Theta Carinae Cluster". It adds none.
The WDS rule changes only "double star" findings: 8 are gone and 124 are new. The check marks a
D*star that Hipparcos does not record as double (no CCDM letter), and a*star that Hipparcos does record as double. The 124 new findings areD*stars whose WDS pairs Hipparcos does not record: optical, wide or faint companions (Canopus, Vega, Capella). By the new rule these are correct. The 8 gone are stars that Hipparcos records as double and that were*(Dschubba, Avior, Sargas, Ascella and 4 others). They are nowD*. One remains: Arcturus, whose only WDS pair is marked dubious.Tests
tests/test_catalog_data.pyreads the committed database, so it changes with it:counts: 22 catalogs, EGC 102, SaM 2,157, Nam 578. IC 1805 is
C+N(Trumpler class "n", fix(catalog-import): type Steinicke rows by status, attach duplicates, mark not-found objects #676).new assertions, with each value read from the new database: M 82's first name is "Cigar Galaxy" and it has no "Bode's Nebulae"; M 57's first name is "Ring Nebula"; NGC 56 has the not-found description and type
?; Mizar is oneD*sky object with Str 56, SaM 2054, WDS 70779 and Nam 65378, magnitudes [2.23, 4.0] and filter magnitude 2.23; Vega has filter magnitude 0.03 and the Hipparcos position of HIP 91262; SaM 1019 has the Hipparcos position of HIP 44804; IC 2145 shares NGC 2086's sky object, NGC 771 shares Nam 9598's, and WDS 99456 shares Nam 94645's (Gumala); Nam 88567's first name is "Dhanuśarāgra".Collinder assertions: Col 399 has "Al Sufi's Cluster", "Brocchi's Cluster" and "Coathanger" as three names, and no "4-5 Vul."; Col 14, Col 189, Col 382, Col 26 and Col 334 share the object that Collinder names; M 44's first name is "Beehive Cluster"; IC 1805 has type
C+N; Col 220 is alone (it is not NGC 3247); Col 142 has the name "NGC 2384a" and is alone; no Collinder name is "nl", has parentheses, has a comma, or is only a number.tests/test_catalog_matching.pyhas a test for each Collinder pattern of the tables above, and a test of the loader with the two files.WDS rule assertions: Atria (an optical pair) and Suhail (a companion 12.6 magnitudes fainter) are
D*; Arcturus (only a dubious pair) is*; Vega isD*; the WDS listings are 120,706D*and 10,597*.tests/test_catalog_star_identity.pytestsis_double_star_pair: an optical pair, a pair more than 5 magnitudes fainter, a pair closer than 0.5″, a pair wider than 60″ and a BC pair are double stars; a dubious (X) pair and a pair with no companion magnitude are not.tests/test_catalog_star_loaders.pytests the loader's types on WDS's own rows.nox: lint, format, type_hints, smoke_tests (5) and unit_tests (2,363) pass.
Docs
docs/ax/catalog.md§9: the "Status (2026-11-08)" note is removed, because the committed database now matches the import that §9 describes. Its list of where the code is, and the link to ADR 0046, stay as a normal paragraph. §9.7 now gives the compressed size as about 24 MB. §9.3 has a new paragraph: how the Collinder loader reads its two name columns. §9.3 also gives the new WDS double-star rule, and the CatalogCONTEXT.mdentry "Double star" agrees with it.release_notes/2.6.4-catalog.mdhas a line "Double stars follow WDS". It compares with release 2.6.3, where every WDS listing wasD*.Related issues
collinder.txt: 7 before, 4 now. The 4 are Col 394 and Col 401 (Collinder's declination errors, corrected incollinder.toml), Col 208 (= IC 2488, withguard = false) and Col 191 (IC 2391, 10.4′).Screenshots
From the app, 128 px. The three WDS screenshots are from the final database. The others are from the Collinder database (
c3cdb493…); the final database changes only star types, so they show the same:miaplacidus-search.pngmizar-search.pngmizar-details.pngnamed-stars-list.pngsac-doubles-names.pngm82-details.pngcol399-details.png,col399-details-2.pngm103-details.pngcoathanger-search.pngwds-doubles-filter.png*before and so not in this listwds-doubles-suhail-in-list.pngsuhail-details.pngOn-device checklist (Rich, before merge)
Done on hardware (rev4) by Rich on 2026-11-19, with the final database (sha256
474d0331…): every item passes. The first start reached the main menu in 13 s; the catalog finished loading in the background after about 1 min 55 s (the cache rebuild).Update a PiFinder to this branch. Let the first start build the catalog cache again. Then:
🤖 Generated with Claude Code